HUB
Sign inSign up
streamhub logo

kavehbc/streamhub

By kavehbc

•Updated 19 days ago

Web-based platform to launch and serve multiple Streamlit/Python apps behind one port

Image
0

128

kavehbc/streamhub repository overview

⁠Streamhub

Streamhub runs many Streamlit and Python web apps (Flask, Dash, FastAPI, …) for many users, each app in its own environment, and serves them all on one port at http://<host>:<port>/<app_id>/. Apps start when first opened, stop when idle or on a schedule, and are listed on boards that can require a login. Everything is configured with JSON files. Streamhub runs on Linux and Windows, and in Docker. Apps can also run as Docker containers or Kubernetes pods.

The full documentation, with screenshots (getting started, user and admin guides, configuration, deployment), is at streamhub.readthedocs.io⁠, built from readthedocs/⁠ (make docs). Maintainer documentation (architecture, every config key, operations, known issues) is in docs/⁠.

Package on PyPI: py-streamhub⁠. Docker image on Docker Hub: kavehbc/streamhub⁠.

⁠Quick start

pip install "py-streamhub[tools,examples]"   # or, in a clone: pip install -e ".[tools,examples]"
streamhub demo          # the demo apps on http://localhost:8500, login admin / admin

Your own hub, in three steps:

  1. streamhub init my-hub asks for a site title, a port and the admin password, and writes a starter config folder.
  2. streamhub run my-hub, then open http://localhost:8500⁠ and sign in.
  3. On the Admin page (account menu → Admin), add your apps, users and settings. Every field has a hint, and lists offer their choices. Nothing needs editing by hand.

You can also skip init, whether you run Streamhub directly or in Docker: on a hub without an admin, http://localhost:8500⁠ opens the setup page. Enter the admin name and password, plus the setup token that the hub printed when it started (Setup token: …). The page tells you where to find it; in Docker, it shows the exact docker logs command. The token makes sure only you can create the admin.

With Docker:

docker run -p 8500:8500 -v streamhub-config:/config kavehbc/streamhub   # then open http://localhost:8500: it guides you through setup
docker run -p 8500:8500 kavehbc/streamhub streamhub demo                # the demo
docker compose up -d --build                                            # or: make docker-up (config in ./config)

In a clone, make install then make demo work too (Windows: make.bat install, make.bat demo); make help lists the other tasks.

⁠Running

streamhub demo [--port N]                                  # the bundled demo, login admin / admin
streamhub init [<folder>] [--yes] [--admin-password …]     # a starter config folder with its admin
streamhub run [--port N] [--ssl-port N] [--debug] [--supervised] [<config folder | streamhub.json>]
streamhub check <config folder>   # validate the configuration without starting anything
streamhub passwd        # hash a password for the users file
streamhub version

The config folder defaults to STREAMHUB_CONFIG, else the current folder. A folder without streamhub.json runs on the defaults, and saving the Settings on the admin page creates the file. python -m streamhub … and python main.py … do the same. Streamhub also comes as a single executable that needs no Python install (how to build it⁠). make start runs the config/ folder and logs to logs/streamhub.log. --debug prints debug logs and disables Slack. --supervised is for service managers (systemd, NSSM, Docker): Restart then exits with code 3 and the manager starts Streamhub again.

⁠Using Streamhub

  • The top menu, on every page, leads to Home, News and each board (a lock marks boards that need a login). On a phone it folds into the ☰ button.
  • Your account menu opens when you click your avatar: the Admin panel sections (Dashboard and Network for managers and admins; Apps and boards, Users, News, Settings, Docker, Static files, Sessions and JSON files for admins), Clear memory, Reload configuration, Restart hub and Stop hub (admins), and Log out. These are built in: the apps file doesn't list them.
  • Home shows your favorites, the boards you can open and the latest news; News lists every announcement.
  • Boards list apps as cards. A colored dot and label show whether an app is running (green), starting (amber), unhealthy (red) or idle (gray). Open starts the app if needed.
  • Search filters the cards of a board by name, ID or description; Running and Favorites narrow them.
  • Favorites: click the star on a card. Favorites come first on their boards and appear on the home page. They are stored in your browser.
  • Dark mode follows your system setting; the ◐ button in the menu switches it.
  • Sign in with a user name and password, or with your company account when single sign-on is set up.
  • Report an issue with the flag on a card: describe the problem and Streamhub sends it to the app's developers.

⁠Configuration in brief

A config folder holds four JSON files. The admin page edits all of them; the full reference is in docs/configuration.md⁠.

  • streamhub.json: port, schedule and time zone of the daily memory clean, branding, timeouts, health checks, notifications (Slack, Teams, email), single sign-on (OpenID Connect, LDAP). Every key is optional.
  • streamhub-apps.json: the first element lists the boards (*.brd; "auth": true requires a login), and every following element is an app.
  • streamhub-users.json: users with a role (admin, manager, user, guest) and the boards they can open. Passwords are stored as hashes only: create one with streamhub passwd, or convert a file with streamhub hash-passwords <config folder>.
  • streamhub-news.json: announcements on the landing page.

Keep secrets out of files: use STREAMHUB_SLACK_HOOK, STREAMHUB_COOKIE_SECRET, STREAMHUB_ADMIN_PASSWORD, STREAMHUB_SMTP_PASSWORD, STREAMHUB_OIDC_CLIENT_SECRET, STREAMHUB_LDAP_BIND_PASSWORD and STREAMHUB_METRICS_TOKEN. Apps don't see them: an app receives only the hub variables listed in its env_passthrough.

⁠App types

"type" decides what Streamhub does with an app, and app_id is any name: the app opens at /<app_id>/.

TypeWhat it is
streamlitStreamlit app (file + folder)
pythonAny Python web server (Flask, Dash, FastAPI, …)
dockerAn image run with Docker: a Streamlit app ("serves": "streamlit") or any web server (web)
kubernetesThe same, as a Kubernetes pod
urlRedirect to url
curlProxy to url, which stays hidden behind Streamhub
pageLink to the hub page /<app_id>/. The admin pages need no entry: they are in the account menu.

Apps files from before 2.2 have no type: the app_id suffix decides (.sl, .py, .url, .curl, and .cmd for the built-in help.cmd), and that still works.

Example:

{
	"board": ["sales.brd"],
	"app_id": "forecast",
	"type": "streamlit",
	"app_name": "Sales forecast",
	"description": "Weekly forecast by region",
	"file": "forecast.py",
	"folder": "/srv/apps/sales",
	"virtual_environment": "forecasting",
	"auto_shutdown": true,
	"auto_start": false,
	"restart": "on-failure",
	"icon": "/cdf/icons/seasons-3.png",
	"developer": [{"name": "Jane Doe", "email": "[email protected]"}],
	"children": []
}

virtual_environment is a conda env name or an environment folder. Other ways to launch an app are python_executable, a command list, or a launcher script in bash_file (.sh, .bat, .cmd or .ps1). To run it from a container image instead, use "type": "docker" (or "kubernetes") with "image" and "serves"; file is then a path inside the image, and volumes mounts folders. Admins can also pull images and add an app on one from the Docker tab. "health_path" names an endpoint that Streamhub polls to detect a hung app and restart it.

⁠Writing apps for Streamhub

  • Streamlit (.sl): nothing to change.

  • Python (.py): Streamhub runs python <file> --port=<port> --app_id=<app_id> (the port is also in STREAMHUB_PORT). Listen on STREAMHUB_HOST (127.0.0.1, or 0.0.0.0 in a container). Give the app a cheap health endpoint and set health_path. The app is served under /<app_id>/ with that prefix removed, and X-Forwarded-Prefix is set:

    # Flask
    app.wsgi_app = ProxyFix(app.wsgi_app, x_for=1, x_proto=1, x_host=1, x_prefix=1)
    # FastAPI
    app = FastAPI(root_path=os.environ.get("STREAMHUB_BASE_PATH", ""))
    
  • The signed-in user is sent to every app as X-Streamhub-User / X-Streamhub-Role (Streamlit: st.context.headers, or st.user with "streamlit_user_headers": true on a recent Streamlit). To be sure a request came through Streamhub, verify X-Streamhub-Identity, a short-lived token signed for your app only:

    import jwt, os  # PyJWT
    claims = jwt.decode(headers["X-Streamhub-Identity"], os.environ["STREAMHUB_APP_KEY"], algorithms=["HS256"],
                        audience=os.environ["STREAMHUB_APP_ID"], issuer="streamhub")
    user, role = claims["sub"], claims["role"]
    
  • Apps don't inherit Streamhub's environment variables. List the ones an app needs in its env_passthrough (for example ["AWS_*"]), or set them in env.

  • Listen on 127.0.0.1 (STREAMHUB_HOST) only; Streamhub warns about apps reachable on other addresses.

Working samples: streamhub/demo/⁠ (what streamhub demo runs).

⁠Internal pages

PathPurpose
/help.cmd/This guide
/news/Every announcement
/admin/Admin panel (manager or admin, also in the account menu): server load, apps with health, memory and CPU, start, stop or restart, live logs, usage today, CPU affinity, listening ports. Admins also edit apps and boards, users, news and settings in forms (a settings save applies at once), manage Docker containers and images, upload and organize the files of /cdf/, and get sessions, the JSON editor and git. /status.cmd/ redirects here.
/out/<app_id>/The app's output log
/clear/Stop apps with auto_shutdown (admins: all apps) and empty the temp folder
/update/Reload streamhub.json (admin)
/restart/, /exit/Restart or stop Streamhub (admin)
/report/?id=<app_id>Report an issue with an app, or contact its developers
/api/Paths and running apps as JSON (admin). Also /api/health and /api/apps (the apps you can open).
/metricsPrometheus metrics (manager, or STREAMHUB_METRICS_TOKEN)
/login/, /logout/Session (/login/oidc/ for single sign-on)
/setup/While the hub has no admin (/ opens it): with the link the hub printed at startup, creates the first admin

⁠Static files

/cdf/<path> serves the static folder (icons, CSS). /temp/<path> serves a public temporary folder (path in /api/ as cdf_temp) that is emptied on every memory clean. Use it for large downloads, and use unique file names.

⁠Troubleshooting

  • "The app stopped": the page shows the end of the app's log. Refresh to start the app again.
  • "The app did not answer … within N s": open /out/<app_id>/ for errors (a missing package, a wrong environment, a wrong port).
  • The report form says your report could not be delivered: try again later, or write to the developers directly.
  • 403 on an app: your user has none of the app's boards.
  • An app restarts by itself: it failed its health checks; the admin page shows its log.
  • Nobody can sign in on a new hub: the login page says so when there is no admin yet. Open the hub's address: the setup page asks for the setup token from the hub's console (docker logs <container> in Docker). Or run streamhub init <config folder>.

More: docs/operations.md⁠.

⁠Reference

This is an extended work based on Streamlit-launchpad⁠.

Tag summary

Content type

Image

Digest

sha256:0aca1e3aa…

Size

208.9 MB

Last updated

19 days ago

docker pull kavehbc/streamhub