SimWrapper Deployment

[cherryqliu]

SimWrapper Deployment

I am reaching out regarding an execution issue I am encountering with the simwrapper package in my local development environment. I am attempting to deploy a configuration (config.yaml) that defines local storage paths (C:, D:, etc.). The server works when launched raw, but the configuration commands fail to start the server process.

I am using:
• SimWrapper Version: 4.1.6
• Operating System: Windows 10 (or specify your Windows version)
• Python Environment: Anaconda Base [Python 3.11]

Command Attempted	Result
cd C:\Apps\simwrapper, then simwrapper run config.yaml	Server starts on Port 4999, Error loading file: \Output\skims\skims.omx.
python -m simwrapper run config.yaml	No module named simwrapper.__main__
simwrapper serve	Server successfully starts on Port 8000 but no config is loaded.
python -m simwrapper.cli run <abs_path>	Returns immediately to prompt (silent failure).
python -m simwrapper.FlaskApp	Server starts on Port 4999, but no config is loaded.

Could you advise on the best approach to ensure the run command correctly initializes and loads the configuration file? It appears there may be an issue with how the entry point scripts are calling the main Python functions.

Thank you for developing SimWrapper—it is a valuable tool, and I look forward to resolving this so I can continue my test.

Best regards,
Cherry

[billyc]

Hello there Cherry, thanks for the question! Let's see if we can get this working for you.

Your first attempt above is the "right way" to start SimWrapper locally. You should be able to cd into your data folder, run the command simwrapper run and then open your browser to localhost:4999 to see all of the folders and files there. They are on the main page but under the "Server" link, I've circled it in pink here:

If that isn't working then we have some debugging to do.

But if that DOES work, then the next step is to get that config.yaml you wrote working too, so that you can view more than one folder. It looks like you have multiple paths you want to view. Can you share that config file here? Maybe we can debug it together.

Note that I am just about to release an updated version of the simwrapper command in the coming days, so if there's a bug here we can get the fix into the next release!

Billy

[cherryqliu]

Hi Billy,

Thanks for the quick response! I really appreciate you looking into this.

I agree, the simple simwrapper run command is the goal. Unfortunately, that is exactly where the issue lies.
As requested, I've attached the content of my config.yaml. It's quite simple, designed only to add the main drive mount points for initial testing.

Best regards,
Cherry

config.yaml

[billyc]

Thanks for sending the file.

From the screenshots, it looks like the app is correctly navigating your folder structure: I can see the "breadcrumbs" at the top showing X:Drive / RTP50_LA25 / Output / Skims. But it can't load the .omx file itself. I can think of two possibilities:

1. Most likely culprit: the file is stored on the network drive "X:" instead of a local drive, and that is causing some sort of loading error. As a test, could you copy that OMX file to either of the shared folders on your C: or D: drives, and see if it opens there? If it does, then I need to figure out why a Windows network share is causing the problem.

2. As an additional test, after copying that OMX file to your C: drive, click the "Matrix Viewer" button on the SimWrapper home page, and then drag the file from Windows Explorer into the Simwrapper window. It should load automatically into the matrix viewer.

3. Maybe there is something unexpected in the OMX file itself; this seems unlikely but maybe its format is different than anticipated. If you are willing to share the OMX file, I could test it here. You can upload it to this fileshare service from TU Berlin:
   https://tubcloud.tu-berlin.de/s/SwQX4ArnToPSz42 -- I will delete the file immediately after our investigation.

Please let me know the results of these tests. And again, if you can find a way to supply me with that OMX file it would be a big help.

Fingers crossed!

[cherryqliu]

Hi Holly,

Thanks for all the great suggestions. I actually tested these scenarios before posting the question - Let me respond to them one buy one:

"the file is stored on the network drive "X:" instead of a local drive, and that is causing some sort of loading error. As a test, could you copy that OMX file to either of the shared folders on your C: or D: drives, and see if it opens there? If it does, then I need to figure out why a Windows network share is causing the problem."

I tried opening an omx file from local C via Data Sources -> Local_C, but the same "Error loading file" message appeared. Interestingly, when I opened a CSV file from the same location, it triggered a different error message, as displayed below.

"As an additional test, after copying that OMX file to your C: drive, click the "Matrix Viewer" button on the SimWrapper home page, and then drag the file from Windows Explorer into the Simwrapper window. It should load automatically into the matrix viewer."

Yes, dragging the file into SimWrapper via the "Matrix Viewer" channel does work—the OMX file loads correctly that way. However, I’d like to be able to configure multiple locations (X:, C:, D:, etc.) for easier access.

At this point, I really doubt the issue lies with the OMX file itself. But if you'd still like to examine it, I could upload it — please let me know.

Best regards,
Cherry

[billyc]

This is all really helpful, thanks for taking the time to explain it. And now I think I have tracked down the problem, and I'm working on a solution. I will let you know when I have something for you to test.

[billyc]

I've uploaded version 4.2.0 to pypi. Please upgrade with

uv tool upgrade simwrapper

and then simwrapper run should work properly. There was a background thread bug that only triggered when accessing files through the Python tool. This version corrects that and should solve your problem. Threading issues are often subtle; sorry this took a couple days to nail down.

This release also includes some other fixes and changes. Let me know how it works for you, especially if the problems continue!

[cherryqliu]

Hi Billy,

Thank you for your efforts updating simwrapper. I've installed version 4.2.0 and it looks like we’ve made some progress.

I was able to successfully load skims.omx the first time I ran this new version, and the logs showed the matrices being sent:

INFO:waitress:Serving on http://0.0.0.0:4999
cannot access local variable 'e' where it is not associated with a value
--- OMX Local_D RTP50_LA25/Output/skims/skims.omx
--- OMX Local_D RTP50_LA25/Output/skims/skims.omx
---sending matrix data COUNTY
--- OMX Local_D RTP50_LA25/Output/skims/skims.omx
---sending matrix data COUNTY
--- OMX Local_D RTP50_LA25/Output/skims/skims.omx
---sending matrix data WLK_LOC_IVT__AM
--- OMX Local_D RTP50_LA25/Output/skims/skims.omx
--- OMX Local_D RTP50_LA25/Output/skims/skims.omx
---sending matrix data WLK_LOC_IVT__AM
--- OMX Local_D RTP50_LA25/Output/skims/skims.omx
--- OMX Local_D RTP50_LA25/Output/skims/skims.omx
---sending matrix data WLK_LOC_IVT__AM
--- OMX Local_D RTP50_LA25/Output/skims/skims.omx
--- OMX Local_D RTP50_LA25/Output/skims/skims.omx
---sending matrix data WLK_LOC_IVT__AM
--- OMX Local_C RTP50_LA25/Output/skims/skims.omx
--- OMX Local_C RTP50_LA25/Output/skims/skims.omx
---sending matrix data WLK_LOC_IVT__AM

However, when I dragged in the geojson file, the loading failed again.
After restarting simwrapper several times, the "Error loading file: ..." recurred and I was unable to load the matrices again:

INFO:waitress:Serving on http://[::]:4999
INFO:waitress:Serving on http://0.0.0.0:4999
cannot access local variable 'e' where it is not associated with a value
--- OMX Local_D RTP50_LA25/Output/skims/skims.omx
---sending matrix data WLK_LOC_IVT__AM

Please let me know if I may have missed a step or if there’s anything I should try to consistently load the matrices.

In the meantime, could you kindly provide steps for other alternatives to deploy SimWrapper or access files outside of the Python tool? I’d like to try those as well.

Thanks again for your help!

[billyc]

Ok I am continuing to investigate this. This is working for me here on all of my test data for multiple cities and on Windows/Mac and various browsers. :-/

But we will solve it.

So if I understand correctly, the site loads, and now you can navigate to a skims.omx file, and when you load that you can see the matrix table view. But when you drag in the boundary file, it fails. That's correct?

Is it possible to zip up and share the GeoJSON file? What is its filename?
If I can't reproduce the problem it's hard to fix it, which is why I ask for the file.

Regarding other methods of using SimWrapper:
(1) The Python approach is still my recommendation, and if it's broken for you then it is broken for other users! So I'd really like to solve your problem. We could also do a videochat if you want to show me what's going on, maybe that would be helpful for me to track it down.

(2) You can also use SimWrapper directly without the Python installer, from https://simwrapper.app. Using Chrome or Edge, click "View Local Files" to navigate to data on a local or network folder. But that method will be noticeably slower than the Python approach, because the browser will have to download the entire OMX file before it can load the visualization. If your OMX files are gigabytes in size like SANDAG, this is going to be unusably slow.

Ultimately when we get this working, I want to add a config option in that YAML file so that you don't need to drag/drop the geojson boundary file anymore -- it will "just work". But let's get this solved first....

[billyc]

I did find one typo in an error message that was causing those "cannot access local variable {e}" messages. I don't think this is the root of the problem, but please upgrade to version 4.2.1 just in case.

Thanks, and apologies for the continued trouble.

[billyc]

This is fixed in release 4.3 and above. I'll mark this as resolved shortly. @cherryqliu if this release is also working for you, please mark this question as answered! Thanks =)

[cherryqliu]

Hi Billy,

Thank you again for your help with deploying SimWrapper. I really appreciate your patience in troubleshooting the issues with me and providing such quick, clear solutions.

Best,
Cherry