Add lightweight netlab API server command #2969

captainpacket · 2025-12-24T17:44:08Z

Adds a minimal netlab API server (stdlib HTTP) to expose CLI actions for automation. Includes new CLI command, help listing, and docs page.

captainpacket · 2025-12-24T17:50:44Z

This is primarily an automation I built to hook netlab into Semaphore UI

DanPartelly · 2025-12-24T19:27:24Z

Thank you ! I certainly see some future in this.

Could you please write some words describing the problem you are trying to solve with this commit? So we can start from a common understanding?

captainpacket · 2025-12-24T19:45:23Z

Engineers here need to model and test lots of different vendors and networks. We've had some internal tooling in the past to do this manually (some hack expect style scripts), but have switched to netlab to make the environment a bit more composable.

We started using Semaphore UI (https://semaphoreui.com/) as a job execution engine, which works well with the built in terraform/ansible worklows in a multi-user environment. It can also do manual bash/python scripts. I'd started with bash scripts with sshpass, but found them kinda brittle. I wanted to switch to an API based workflow to make it a bit more robust.

I'd initially wrote a FastAPI based wrapper that imports these functions (which I'm still using), but I had the thought that this could be easily extended in the native CLI workflow. A typed version I think still has use and this could be extended for that.

ipspace · 2025-12-26T08:32:20Z

@captainpacket -- Thanks a million for the PR. Definitely an interesting idea ;) and I truly appreciate you crossing all the Ts and dotting all the Is.

Give me a few days -- I'm stuck in the "we have to deal with another Ansible brokenness" bog, and have to get out of that first.

captainpacket · 2025-12-28T12:59:39Z

Cleaned up a few semaphore specific things (project/task) that I'd left behind; added minimal authentication and TLS option

ipspace

Mostly minor details and improvement suggestions, the only showstopper is whether jobs can be run in parallel and how to handle that.

ipspace · 2025-12-30T08:10:29Z

docs/netlab/cli.md

   netlab config <config.md>
   netlab connect <connect.md>
   netlab create <create.md>
+   netlab api <api.md>


I think this list is sorted, so "netlab api" should be at the very top

Good catch — moved netlab api to the top so the list stays sorted.

ipspace · 2025-12-30T08:11:48Z

netsim/cli/api.py

+from . import status as netlab_status
+from . import up as netlab_up
+
+DATA_DIR = os.getenv("NETLAB_API_DATA_DIR", "/var/lib/netlab/api")


I don't think this is user-writeable on most systems, so the HTTP server would have to be started as root. I usually use "/tmp/something" in these cases (or you could use "~/...")

Agreed — changed the default data/log dir to a user-writable temp location; still overrideable with NETLAB_API_DATA_DIR.

ipspace · 2025-12-30T08:12:51Z

netsim/cli/api.py

+  handler.wfile.write(data)
+
+
+def require_auth(handler: BaseHTTPRequestHandler) -> bool:


Just wondering: isn't there a Python library to handle this?

I couldn’t find a clean stdlib helper for server-side Basic Auth parsing; kept stdlib-only to avoid extra deps and refactored it into a small helper to keep the handler clean.

ipspace · 2025-12-30T08:17:26Z

netsim/cli/api.py

+def workspace_dir(payload: Dict[str, Any]) -> str:
+  workdir = (payload.get("workdir") or "").strip()
+  if workdir:
+    if os.path.isabs(workdir):


It looks to me like you're trying to get an absolute path to the working directory. If that's the case you could probably use payload.get("workdir",payload.get("workspaceRoot",os.getcwd())) to get the path you want to get or loop through the keywords instead of effectively having the same code twice.

Also, I'd usually use pathlib.Path(path).resolve() to get an absolute path.

Refactored workdir/workspaceRoot selection and switched to pathlib.Path(...).resolve() for absolute paths; removed duplicated branches.

ipspace · 2025-12-30T08:18:12Z

netsim/cli/api.py

+
+
+def resolve_topology(payload: Dict[str, Any], workdir: str) -> str:
+  topology_url = (payload.get("topologyUrl") or "").strip()


As you're using the same expression so many times, it might make sense to turn it into a function

Done — factored the repeated path/payload handling into helper functions.

ipspace · 2025-12-30T08:21:52Z

netsim/cli/api.py

+  results: List[Dict[str, str]] = []
+  if not template_dir:
+    return results
+  for root, dirs, files in os.walk(template_dir):


It looks like you're doing list(glob('**/*.yml')) + list(glob('**/*yaml')). You could use glob from os.path or from pathlib

Switched template discovery to pathlib globbing for .yml/.yaml

ipspace · 2025-12-30T08:24:05Z

netsim/cli/api.py

+      _run_with_output(netlab_down.run, args)
+    elif action == "collect":
+      args = []
+      if payload.get("instance"):


I would probably do this with a dictionary mapping payload variable names into netlab args, and use the same approach for all commands.

Agreed — refactored payload→argv construction to a consistent helper-based approach across actions.

ipspace · 2025-12-30T08:27:33Z

netsim/cli/api.py

+    os.chdir(prev_cwd)
+
+
+def start_job(payload: Dict[str, Any]) -> Dict[str, Any]:


Can you start jobs in parallel? If that's the case, you need multilab plugin, or you have to serialize the jobs.

Good point; I’m serializing job execution (queueing requests but only running one netlab action at a time). If we later want true parallel jobs, we should detect/require multilab and unique ID per job rather than reinventing isolation.

ipspace · 2025-12-30T08:28:55Z

netsim/cli/api.py

+      send_json(self, HTTPStatus.NOT_FOUND, {"error": "not found"})
+      return
+
+    if parts == ["healthz"]:


I would usually implement something like this with a dictionary mapping keywords (parts in you case) into functions that do the actual work.

Refactored routing to a dict mapping endpoints/keywords to handler functions (kept the /jobs//... special-case)

ipspace · 2025-12-30T08:29:58Z

netsim/cli/api.py

+
+
+def run(cli_args: List[str]) -> None:
+  parser = argparse.ArgumentParser(description="netlab API server")


All other CLI commands have parser in a separate function to keep the "run" function shorter (and cleaner)

Aligned with other CLI modules — moved parser construction into a separate api_parse_args() function so run() stays short.

captainpacket force-pushed the skyforge-api-server branch 5 times, most recently from 61a1f3d to 76d3284 Compare December 26, 2025 13:25

feat: add netlab api server command

57e732e

captainpacket force-pushed the skyforge-api-server branch from 76d3284 to 57e732e Compare December 26, 2025 13:30

ipspace requested changes Dec 30, 2025

View reviewed changes

Refine netlab api server review feedback

77051d6

		handler.wfile.write(data)


		def require_auth(handler: BaseHTTPRequestHandler) -> bool:



		def resolve_topology(payload: Dict[str, Any], workdir: str) -> str:
		topology_url = (payload.get("topologyUrl") or "").strip()

		os.chdir(prev_cwd)


		def start_job(payload: Dict[str, Any]) -> Dict[str, Any]:



		def run(cli_args: List[str]) -> None:
		parser = argparse.ArgumentParser(description="netlab API server")

Add lightweight netlab API server command #2969

Are you sure you want to change the base?

Add lightweight netlab API server command #2969

Conversation

captainpacket commented Dec 24, 2025

Uh oh!

captainpacket commented Dec 24, 2025

Uh oh!

DanPartelly commented Dec 24, 2025

Uh oh!

captainpacket commented Dec 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ipspace commented Dec 26, 2025

Uh oh!

captainpacket commented Dec 28, 2025

Uh oh!

ipspace left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

captainpacket Jan 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

captainpacket Jan 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

captainpacket commented Dec 24, 2025 •

edited

Loading

captainpacket Jan 2, 2026 •

edited

Loading

captainpacket Jan 2, 2026 •

edited

Loading