170 add extractor to gui #184
Conversation
working on adding submit dataset to extractor
also adding a sample extractor_info.json file for registration purposes
…-extractor-submit-add-submit-dataset # Conflicts: # backend/app/routers/datasets.py
beginning of adding listener/extractor to front end
# Conflicts: # backend/app/models/extractors.py # backend/app/models/feeds.py # backend/app/models/listeners.py # backend/app/models/search.py # backend/app/rabbitmq/listeners.py # backend/app/routers/datasets.py # backend/app/routers/feeds.py # backend/app/routers/files.py # backend/app/routers/listeners.py # backend/app/tests/test_extractors.py # frontend/src/openapi/v2/services/DatasetsService.ts
|
In order to test this, you will need to use the pyclowder from this pull request: Files and datasets can be submitted. |
# Conflicts: # backend/app/routers/metadata_files.py # frontend/src/components/Explore.tsx # frontend/src/components/Layout.tsx # frontend/src/components/datasets/Dataset.tsx # frontend/src/components/files/File.tsx # frontend/src/openapi/v2/services/DatasetsService.ts
|
Instructions on how to test. This pull request depends on this pull request in pyclowder. clowder-framework/pyclowder#51 Without using that pyclowder, the extractors will not work with clowder v2. To register an extractor: create a new configuration to run the script: /app/rabbitmq/heartbeat_listener_sync.py I had issues with imports from clowder2, but if I opened it up in /backend in PyCharm I was able to run it without issues. While that is running, start an extractor (example - wordcount). You will then see that it is registered in the console. After that point, you don't need to run the heartbeat monitor. If you run clowder, you can see the extractors/listeners under the page: localhost:3000/listeners You can submit a file or dataset to an extractor. There is a button for each of them for EXTRACT. Right now the GUI is pretty bad, and it's not possible to support parameters through the GUI even though the backend routes support them. Also, every now and then pyclowder will throw an error 'host is empty this is bad.' Restarting everything should fix that. I will make that a separate issue. Additionally, right now the metadata for the extractor is not visible under the 'metadata' tab. Looking for input as to how to display it and where. |
|
|
||
| // parameters | ||
| const search = useLocation().search; | ||
| const {datasetId} = useParams<{ datasetId?: string }>(); |
There was a problem hiding this comment.
let [searchParams, setSearchParams] = useSearchParams();
const datasetId = searchParams.get("datasetId");
const fileId = searchParams.get("fileId");
@longshuicy go for it, please squash |
No description provided.